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JANUARY  14, 1983 


The  Honorable  Caspar  W.  Weinberger 
The  Secretary  of  Defense 

Attention:  Director,  GAO  Affairs 

Dear  Mr,  Secretary: 

Subject:  Greater  Benefits  to  Be  Gained  From  DOD 
Simulators  (GAO/FPCD-83-4 ) 


Flight 


The  Department  of  Defense  has  adopted  the  policy  of  using 
simulators  whenever  possible  to  reduce  training  costs  and  to 
improve  training  effectiveness.  Simulators  offer  opportuni¬ 
ties  to  improve  training  capability;  extend  a  weapon  system's 
life;  reduce  accidents;  and  reduce  range,  fuel,  ammunition, 
and  missile  requirements.  Defense  has  invested  over  a  billion 
dollars  in  flight  simulators  and  plans  to  spend  an  estimated 
$3.2  billion  over  the  next  5  years  for  new  flight  simulators, 
modifications  to  and  additional  copies  of  existing  simulators, 
and  spare  parts. 

Whether  this  large  investment  will  reduce  training  costs 
and  will  meet' the  services'  future  training  needs  depends  on 
how  well  the  services 


— analyze  their  training  needs, 

— design  simulators  specifically  to  meet  their  needs,  and 

— develop  training  plans  incorporating  simulators  into 
their  training  programs. 

We  found  that  the  services  are  not  always  analyzing  their 
training  needs  and  thus  do  not  know  what  tasks  can  best  be 
taught  on  simulators.  In  addition,  the  services  are  not 
always  incorporating  simulators  into  their  training  programs. 
As  a  result,  the  simulators  are  not  being  used  as  effectively 
as  they  could  be.  These  problems  can  adversely  affect  the 
potential  benefits  from  the  services'  planned  investment  in 
flight  simulators.' 
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OBJECTIVES ,  SCOPE,  AND  METHODOLOGY 
" — r^Our  objectives  were  to  determine  whether  j 

-^the  services  are  sufficiently  analyzing  their  training 
needs  and  identifying  those  tasks  that  can  best  be 
taught  on  flight  simulators^  and  j 

-Simulators  are  training  pilots  to  do  the  tasks 
intended. 

^ — . ....  . y 

t£>-  m>bt  q$:  obj^tivesv  we  selected  seven  flight 
simulators  that^ 

-Sere  among  the  services'  highest  20  percent  in  terms 
of  total  dollars  invested^ 

-- ^ero  used  to  train  personnel  for  critical  job,-*  (tasks 
that  contribute  substantially  to  readinessJJ  a.id, 

(T‘  .  -  ■  -  . 

--had  been  fully  operational  for  at  least  2  years. 

~^The  locations  we  visited  were  in  geographical  areas  with 
large  concentrations  of  simulators.  At  each  location,  we 
examined  course  syllabi  to  determine  how  the  simulators  were 
to  ’j»s  used.  We  also  determined  how  the  simulators  were  teing 
used,  benefits  derived,  and  problems  with  the  simulators. 

The  following  table  lists,  by  service,  the  simulators 
chosen  for  review,  their  costs,  and  the  training  centers  and 
operating  units  which  we  visited. 


Servioe 

Navy 

Simulator 

Air  combat  manuevering 
simulator-2E6  <P-4,  F-14) 

Total 

number 

1 

Total 

ooet 

(mill  ions) 

$  17.5 

Locations 

Oceana  Naval  Air  Station 
Virginia  Beach,  Va. 

Operational  flight  trainer-2F95 
(M4) 

4 

26.7 

Miramar  Naval  Air  Station 
San  Diego,  Calif. 

Air  Fox* 

Operational  flight  trainer,  M5 

a/ 11 

2/81.1 

Langley  Air  Ft*ce  Base, 
Hmnptcn,  va>  Luke  Air 

Force  Base,  Phoenix,  Ariz. 

Operational  flight  trainer,  F-4E 

12 

2/30.0 

Seymour  Johnson  Air  Force 
Base,  Goldsboro,  N.C. ; 

Luke  Air  Force  Base, 
Phoenix,  Ariz. 

Marina  Corps 

Weapon  aystam  trainer-2F88  (F-4) 

3 

18.3 

Yuma  Marine  Corps  Air 
Station,  Yuma,  Ariz. 

Operational  flight  trainer 

2F117  (CH-46) 

1 

10.3 

Marine  Corps  Air  Station, 
New  River,  N.C. 

Army 

Trainer  synthetic  flight  aye  teen  ®-l 

22 

57.0 

Port  Rucks r,  Ala;  Fort 
£'■11,  Lawton,  Okie. 

y  Additional  information  provided  in  Defense's  oonmenta,  which  was  not  validated  by  GAO. 


2 


B-208653 


We  conducted  a  literature  search  and  contacted  defense 
agency  audit  groups  to  examine  previous  studies  on  simu¬ 
lators.  We  also  contacted  the  following  organizations  to  find 
out  how  each  service  manages  simulators  and  how  much  the 
services  plan  to  spend  on  simulators  over  the  next  5  years. 

Army 

Deputy  Chief  of  Staff  for  Operations  and  plans 
Training  and  Doctrine  Command 

Training  and  Doctrine  Command's  Systems  Support  Analysis 
Activity 

Army  Training  Support  Center 

Office  of  Project  Manager  for  Training  Devices 
Army  inspector  General 

Navy 

Deputy  Chief  of  Naval  Operations,  Chief  of  Naval 
Personnel 

Deputy  Chief  of  Naval  Operations,  Aviation  Manpower  and 
Training 

Naval  Air  System  Command ,  Headquarters 
Naval  Training  Equipment  Center 

Fleet  Aviation  Specialized  Operational  Training  Group, 
Atlantic 

Naval  Audit  Service 
Marine  Corps 

Deputy  Chief  of  Staff  for  Training,  Policy  Control  Branch 
Deputy  Chief  of  Staff  for  Aviation,  Aviation  Weapons 
Systems  Requirement  Branch 

Air  Force 

Deputy  Chief  of  Staff  for  Operations,  Plans  and 
Reediness,  Rated  Management  Division 
Deputy  Chief  of  staff,  Research,  Development  and  Acquisi¬ 
tion 

Air  Force  joint  Cruise  Missiles  project  Office 
Tactical  Air  Warfare  Center 

4444  Operational  Squadron,  Tactical  Air  Command 

We  made  these  visits  between  November  1981  and  June  1982, 
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TRAINING  NEEDS  ARE  NOT  ANALYZED 


Before  deciding  on  what  type  of  simulator  to  use  for 
training,  the  services  must  first  analyze  their  training 
needs.  Such  an  analysis  involves  (1)  identifying  tasks  that 
need  to  be  taught  and  (2)  choosing  the  best  option  to  teach 
these  tasks. 

We  found  that  the  services  did  not  analyze  their  training 
needs  for  the  simulators  selected  for  review.  Instead,  they 
focused  on  duplicating  the  actual  weapon  systems  and  the  it 
surroundings.  In  reviewing  documents  justifying  the  purchase 
of  flight  simulators,  we  found  descriptions  only  of  what  the 
simulators  should  do — which  was  primarily  to  replicate  the 
actual  equipment — with  little  reference  to  how  the  devices 
could  meet  training  needs. 

This  emphasis  on  equipment  capability,  rather  than  the 
training  needs  that  can  best  be  met  by  simulators,  has  been 
recognized  in  defense  agency  audit  reports.  The  Naval  Audit 
Service,  in  a  January  1980  report,  V  questioned  whether  the 
Navy  had  adequately  assessed  its  training  needs  before  buying 
simulators,  citing  the  2E6  air  combat  maneuvering  simulator  as 
an  example.  Also,  the  Army  Audit  Agency,  in  a  March  1982 
report,  £/  pointed  out  that  the  Army  had  not  used  training 
needs  as'a  basis  for  designing  its  UH-1  synthetic  flight, 
simulator.  Instead,  the  Army  attempted  to  design  a  simulator 
that  would  replicate  all  possible  flight  characteristics. 

SIMULATORS  ARE  NOT  ALWAYS 
INCORPORATED  INTO  TRAINING  PROGRAMS 

To  insure  that  simulators  are  used  as  intended,  the 
services  should  incorporate  them  into  training  plans. 

Although  training  units  we  visited  for  new  pilots  2/  had 
included  simulators  in  specific  training  syllabi,  operating 


J/"Servicewide  Audit  of  the  Aircraft  Flight  Simulator 

Program,"  Audit  Report  140058,  Naval  Audit  Service  Southeast 
Region,  Virginia  Beach,  Va. 

2/"Synthet ic  Flight  Training  System,"  Audit  Report  S082-6, 

”  U.S.  Army  Audit  Agency,  Washington,  D.C. 

2/New  pilots  include  those  (1)  coming  out  of  basic  training, 
(2)  making  a  transition  to  other  aircraft,  and  (3)  returning 
to  an  aircraft  after  having  not  flown. 
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units  had  no  such  syllabi  for  training  experienced  pilots. 

For  example,  the  Navy's  F-14  and  the  Air  Force's  F-15 
operational  flight  trainers  had  been  incorporated  into  the 
training  course  syllabi  for  new  pilots,  but  the  operating 
units  reviewed  had  estaj--1  ished  only  minimum  hour  requirements 
for  simulator  use,  leaving  the  training  content  up  to  the 
individual  in  training.  The  Air  Force  tactical  squadrons 
visited  required  each  experienced  F-15  pilot  to  use  the 
trainer  only  12  hours  a  year;  the  Navy  fleet  squadrons  visited 
had  no  minimum  hour  requirement  for  F-14  pilots.  Operating 
units  of  both  services  used  their  trainers  primarily  for 
periodic  proficiency  qualifications  preparation  and  testing. 

INADEQUATE  NEEDS  ASSESSMENT  AND 
TRAINING  PLANS  MAY  ADVERSELY 
AFFECT  SIMULATOR  TRAINING 

Because  the  services  do  not  always  analyze  their  training 
needs  nor  incorporate  simulators  into  training  programs,  they 
are  left  with  systems  that  may  not  meet  their  needs  and  that 
may  not  be  used  as  effectively  as  they  could  be.  Two  examples, 
are  the  Navy's  2E6  air  combat  maneuvering  simulator  and  2F95 
operational  flight  trainer. 

The  2E6,  Which  costs  about  $18  million,  is  an  aerial 
combat  maneuvering  trainer  for  both  the  F-4J  and  F-14A 
aircraft.  It  can  simulate  1-on-l  or  2-on-l  aerial  combat 
environments,  while  providing  pilots  340-degree  visual 
capability.  The  2E6  can  also  simulate  battles  with  various 
adversary  aircraft.  Yet,  the  Navy's  Atlantic  Air  Command  uses 
the  2E6  primarily  to  provide  new  pilots  basic  air-to-air 
tactics  training,  utilization  averaged  only  about  4  hours  a 
day  for  each  of  the  simulator's  two  cockpits  for  calendar  year 
1981.  The  Navy,  about  3  years  after  fielding  the  2E6,  is 
being  assistsd  by  a  contractor  to  identify  what  tasks  can  best 
be  taught  on  the  2E6  and  to  develop  a  training  syllabus. 

The  Navy's  Pacific  Air  Command  has  questioned  the  need 
for  the  2E6.  Although  the  2E6  was  orginally  scheduled  to  be 
placed  at  both  Atlantic  and  Pacific  facilities,  the  Pacific 
Air  Command  expressed  doubt  as  to  the  training  effectiveness 
of  the  2E6  and  believed  equivalent  training  could  be  obtained 
from  other  simulators.  For  these  reasons,  according  to  a 
Pacific  Air  Command  official,  the  command  did  not  receive  and 
has  not  requested,  a  2E6. 
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The  2F95,  at  Miramar  Naval  Air  Station  in  San  Diego, 
costs  $6.7  million  and  is  an  operational  flight  trainer  for 
the  F-14  aircraft.  It  simulates  both  motion  and  visual 
characteristics,  similar  to  the  actual  aircraft,  and 
familiarizes  pilots  with  operational  procedures,  navigation, 
communications,  and  mission  operations,  as  well  as  flight 
system  malfunctions.  The  2F95  can  be  linked  with  an  F-14  rear 
seat  trainer  to  provide  complete  aircrew  training,  including 
certain  tactics.  Yet,  reportedly,  the  F-14  training  squadron 
uses  the  2F95  primarily  for  basic  instrumentation  and 
procedures  training.  The  fleet  F-14  squadrons  also  make 
limited  use  of  the  2F95,  mostly  for  pilots'  practice  before 
flight  qualifications  tests  in  the  actual  aircraft.  An 
official  from  one  of  the  fleet  squadrons  we  visited  said 
required  use  of  the  2F95  had  not  been  made  a  part  of  the 
squadron's  training  program  because  the  trainer  did  not 
specifically  meet  fleet  squadron  training  needs.  As  a  result, 
the  2F95  has  been  used  less  than  50  percent  of  the  available 
training  time. 

NOTED  PROBLEMS  MAY  HAVE  IMPACT 
UPON  FUTURE  SIMULATORS 

The  services  reportedly  are  continuing  to  experience 
problems  with  determining  simulator  requirements  and  using 
simulators.  Therefore,  the  problems  we  noted  with  simulators 
currently  in  use  will  likely  be  found  in  the  development  and 
use  of  new  simulators. 

Service  regulations  require  an  analysis  of  training  needs 
to  justify  buying  simulators.  Nevertheless,  we  found  that 
training  needs  are  not  always  adequately  analyzed  to  form  the 
criteria  for  developing  new  simulators.  For  example,  a  Naval 
Training  Equipment  Center  official  said  that  two  simulators 
being  planned,  the  TH-57  helicopter  trainers  (flight 
instrument  and  cockpit  procedures  trainers)  and  the  EA-6B 
cockpit  procedure  trainers,  have  had  hardware  decisions  made 
without  training  needs  being  analyzed.  Officials  from  other 
services  also  expressed  concern  over  the  lack  of  timely  and 
adequate  analysis  of  training  needs  for  designing  and 
developing  new  simulators. 

In  addition  to  purchasing  new  simulators,  the  services 
are  planning  to  buy  additional  copies  of  existing  simulators. 
The  Navy's  plan  to  buy  another  2E6,  estimated  to  cost  $23.3 
million,  is  of  particular  concern  because  training  needs  were 
not  analyzed  for  the  first  2E6  nor  has  the  simulator  been 
fully  incorporated  into  operating  units'  training  programs. 
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Additionally,  according  to  Navy  officials,  a  user  activity  for 
the  planned  2E6  has  not  been  identified. 

CONCLUSIONS 


Simulators  are  becoming  an  increasingly  important  element 
of  military  training  but,  similar  to  the  systems  they  are 
simulating,  they  are  becoming  more  costly.  To  insure  greater 
benefits  from  their  large  investment  in  simulators,  the  ser¬ 
vices  should  develop  simulators  to  meet  proven  training  needs 
and  incorporate  them  into  specific  training  programs.  Our 
review  indicates  that  the  services  are  not  always  doing  this, 
as  specifically  demonstrated  by  the  Navy's  2E6  air  combat 
maneuvering  simulator. 

RECOMMENDATIONS 


We  recommend  that  the  Secretary  of  Defense: 

— Approve  budget  requests  for  flight  simulators  only 
after  the  services  have  analyzed  their  training  needs 
and  proven  that  the  needs  cannot  be  met  with  existing 
simulators.  (Specific  review  should  be  made  of  the 
pending  purchase  of  an  additional  2E6.) 

— Require  the  services  to  incorporate  simulators  into 
their  training  programs. 

AGENCY  COMMENTS  AND  OUR  EVALUATION 

In  commenting  on  our  draft  report,  Defense  agreed  in 
principle  with  our  conclusions  and  recommendations  but  dis¬ 
agreed  with  most  of  the  data  in  the  report.  Basically, 

Defense  asserted  that  the  services  did  analyze  specific  train¬ 
ing  needs  for  the  selected  simulators  and  did  include  the 
devices  in  training  plans  for  operational  units.  However,  in 
following  up  on  the  material  presented  by  Defense,  we  did  not 
find  support  for  its  contentions.  The  enclosure  contains 
Defense's  comments  and  our  detailed  evaluation  of  those 
comments. 


As  you  know,  section  236  of  the  Legislative  Reorgani¬ 
zation  Act  of  1970  requires  the  head  of  a  Federal  agency  to 
submit  a  written  statement  on  actions  taken  on  our  recommenda¬ 
tions.  This  written  statement  must  be  submitted  to  the  House 
Committee  on  Government  Operations  and  the  Senate  Committee  on 
Governmental  Affairs  not  later  than  60  days  after 
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RCWRVC  AFFAIRS 
ANO  LOGISTICS 


ASSISTANT  SECRETARY  OF  DEFENSE 
WASHINGTON.  D.C.  *0301 


5  NOV  *82 


Mr.  Clifford  I.  Gould 
Director ,  federal  Personnel  and 
Compensation  Division 
U.S.  General  Accounting  Office 
Washington ,  D.C.  20548 

Dear  Mr.  Gould: 

Thank  you  for  your  letter  to  the  Secretary  of  Defense ,  dated 
October  1,  1982,  concerning  your  draft  report,  "Greater  Benefits 
to  be  Gained  From  DoD  Flight  Simulators,"  OSD  Case  #6106,  GAO 
Code  967025. 

The  Department  concurs  in  principle  with  the  conclusions  and 
recommendations  stated  in  the  draft  report.  Analysis  of  the  need 
for  additional  flight  simulators  is  done  as  a  part  of  the  program 
and  budget  review  process  conducted  within  the  Department  of 
Defense.  Justification  for  simulator  procurement  is  based  on 
training  needs  identified  by  each  Service.  In  the  specific  case 
of  the  Navy's  2E6  trainer,  the  Navy  will  review  and  update  their 
plans  and  justification  for  the  2E6  during  the  next  two  annual 
program  reviews.  Future  budget  decisions  regarding  the  2E6  will 
be  made  subsequent  to  the  completion  of  the  program  review 
process. 

The  Department  believes  that  analysis  of  training  needs  and  ' 
documentation  of  those  needs  for  flight  simulators  are  continuing 
to  improve  through  refinements  of  the  Instructional  Systems 
Development  (ISD)  process.  Incorporation  of  simulators  into  the 
training  plan  for  pilots  is  an  integral  part  of  the  ISD  process. 
Further,  the  Services  have  identified  specific  simulator  training 
that  is  required  for  pilots.  As  a  result,  pilot  training 
programs  have  become  better  and  the  readiness  of  our  forces  has 
Increased.  A  detailed  response  to  the  findings,  conclusions  and 
recommendations  is  enclosed. 


Thank  you  for  your  interest  in  this  matter. 


Enclosure 


Sincerely, 


^^4 +  *{/ 

Jamet  M.  intfene 

Principal  Deputy  Ault**'??  Sscr***^*  r*  Qefa 
{Manpower,  Rtsuve  kV.Utt  | 1  "1“^ 


GAO  notaai  Page  reference*  have  been  changed  to  correspond  with 
those  in  the  report. 


GAO's  coaments  follow  each  of  Defense's  summary  of 
findings  and  responses  on  pages  10  through  19. 
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GAO  DRAFT  REPORT  FPCD-83-4 
OSD  Case  No.  6106 

SUMMARY  OF  FINDINGS,  CONCLUSIONS  AND  RECOMMENDATIONS 


FINDINGS: 

1.  Defense  has  Invested  over  a  billion  dollars  in  flight 
simulators  and  plans  to  spend  an  estimated  $3.2  billion  over  the 
next  five  years  for  new  flight  simulators,  modifications  to  and 
additional  copies  of  existing  simulators,  and  spare  parts. 

(P9*  1) 

DoD  Position:  Concur. 

2.  GAO  found  that  the  Services  did  not  analyze  their 
training  needs  for  the  simulators  but  instead  focused  on 
duplicating  the  actual  weapon  system  and  their  surroundings. 

(pg.  4) 

DoD  Position:  Nonconcur.  The  Services  have  analyzed  the 
specific  training  needs  for  the  instruction  of  military  pilots. 
These  training  needs  have  been  the  basis  for  identifying  the 
specific  capabilities  required  in  a  simulator  to  ensure  effective 
training.  In  several  cases,  the  Services  have  procured  part  task 
trainers  for  those  training  tasks  that  can  be  isolated.  Since 
the  late  1970's,  the  Services  have  further  improved  training 
analysis  and  documentation  of  training  needs  under  the 
Instructional  Systems  Development  (ISD)  process. 

In  order  for  the  simulator  to  be  effective  for  many  pilot 
training  tasks,  it  must  properly  represent  the  aircraft  and  its 
equipment.  The  amount  of  fidelity  required  in  a  simulator  to 
have  effective  training  is  a  complex  question  that  has  not  been 
answered  by  the  research  community.  The  extent  to  which  the 
Services  duplicate  hardware  for  effective  training  is  consistent 
with  the  policier  of  experts  in  civil  and  commercial  aviation. 
While  research  continues  on  this  issue,  the  Services  will 
continue  to  analyze  and  identify  training  tasks  that  are  best 
taught  in  a  simulator  and  those  that  can  be  taught  through  other 
modes  of  instruction. 

The  current  Navy  directives  which  provide  policy  and  procedures 
for  establishing  requirements  and  establishing  qualitative  and 
quantitative  inputs  are: 

OPNAVINST  1500. 8J  -  Subj :  "Navy  Training  Planning 
Process  in  Support  of  New  Development" 
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OPNAVINST  1551. 7B  -  Subj :  "Fleet  Participation  in 
Development  Acquisition,  and  Acceptance  of  Major 
Training  Devices" 

OPNAVINST  1500. 11G  -  Subj:  "Naval  Aviation  Train¬ 
ing  Program  Policies,  Responsibilities,  and 
Procedures" 

GAO  Comment 


Defense*  comments  that  the  services  have  analyzed 
the  specific  training  needs  for  the  instruction  of 
military  pilots  misleads  the  reader.  Whereas  the  cur¬ 
rent  procedures  for  approving  the  purchase  of  flight 
simulators  require  analysis  of  training  needs  as  part 
of  the  ISD  process,  such  assessments  were  not  required 
when  the  systems  we  reviewed  were  approved  for  pur¬ 
chase.  Further,  the  adequacy  of  the  current  proce¬ 
dures  for  determining  training  needs,  as  well  as  the 
adequacy  of  the  analysis  resulting  from  their  applica¬ 
tion,  has  not  been  demonstrated.  These  questions  are 
being  addressed  in  a  current  GAO  review. 

Regarding  the  amount  of  fidelity  (properly  repre¬ 
senting  the  aircraft  and  its  equipment)  required,  we 
recognize  that  more  research  needs  to  be  done,  and 
that  high  fidelity  for  many  attributes  is  very  expen¬ 
sive.  Thus,  we  encourage  the  services  to  analyze  the 
tasks  that  need  to  be  taught  and  include  only  those 
features  the  services  know  are  necessary. 

Using  the  hardware  fidelity  requirements  of  civil 
and  commercial  aviation  as  support  for  the  services' 
action  is  inappropriate  for  several  reasons.  For 
instance,  fidelity  in  hardware  is  not  the  only  nor  the 
most  expensive  item  being  duplicated.  The  behavior  of 
the  aircraft,  along  with  the  visual  capabilities  and 
other  cues,  can  be  more  expensive  than  hardware  fidel¬ 
ity.  Additionally,  the  services  recognize  the  vast 
difference  in  the  missions  of  commercial  aviation  and 
tactical  aircraft,  as  well  as  the  policies  on  flight 
hour  substitution,  which  makes  required  simulation 
very  different. 

In  its  third  point.  Defense  cites  several  Navy 
regulations.  We  agree  that  the  Navy  has  regulations 
and,  in  fact,  recognize  that  all  the  services  have 
these  regulations.  Nevertheless,  having  regulations 
does  not  guarantee  they  are  being  implemented  as 
intended,  and  Defense  provided  no  additional  evidence 
to  demonstrate  the  regulations  were  being  implemented. 
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3.  GAO  found  that  although  training  units  had  included 
simulators  for  specific  training,  operating  units  had  not 
included  them  for  training  experienced  pilots.  (pg.  4-5) 

DoD  Position:  Nonconcur.  Each  of  the  Services  require  the  per¬ 
formance  of  specific  training  to  be  conducted  in  simulators  by 
experienced  pilots  assigned  to  operational  units.  The  Army 
requires  a  minimum  of  20  simulator  hours  a  year  for  each  of  its 
pilots  stationed  at  a  facility  with  simulator  support. 
Additionally,  the  Air  Crew  Training  Manual  (ATM)  stipulates 
maneuvers  to  be  performed  in  the  simulator  for  each  aircraft. 

The  Navy  uses  operational  flight  trainers  for  periodic 
proficiency  qualification,  preparation  and  review  for  experienced 
pilots.  Current  Navy  directives  pertinent  to  this  policy  are 
COMNAVAIRLANTINST  3500. 42E,  AIRLANT  Ltr.  Ser  0365  of  9  June  1980, 
and  COMF1 TWINGONEINST  3500. 4A,  3710. 3C.  Additionally,  formal 
syllabi  for  fleet  pilots  are  in  use  for  the  2E6  Air  Combat 
Maneuvering  Trainer  at  FITWINGONE,  Oceana,  Va. 


In  the  Air  Force,  Tactical  Air  Command  (TAC)  regulation  51-9, 
Employment  of  Aircrew  Training  Devices,  requires  that  simulator 
training  officers  prepare  lesson  plans  for  all  simulator 
training.  These  lesson  plans  tailor  the  simulator  training  to 
best  meet  the  operational  role  of  each  wing.  At  Langley  AFB, 
nine  separate  simulator  lesson  plans  have  been  developed  to 
simulate  world-wide  air-to-air  employment.  Operational  F-15 
pilots  are  required  by  TAC  regulation  51-50,  Flying  Training,  to 
train  18  hours  per  year  in  the  simulator.  According  to  the 
training  records  documented  in  the  TAC  Automated  Flying  Training 


Management  System  (TAFTRAMS)  ,  F-15  pilots  at  Langley  f.FB  trained 
an  average  of  22  hours  per  year  in  the  simulator  while  Instructor 
pilots  at  Luke  AFB  averaged  over  60  hours  per  year  instructing 
and  training  in  the  simulator. 

GAO  Comment 

Although  we  did  not  include  the  Army  in  o„r  dis¬ 
cussion  on  page  4,  our  information  supports  that 
pilots  stationed  at  an  Army  facility  with  simulators 
are  required  to  use  the  simulators  20  hours  a  year. 

Our  review  of  the  Aircrew  Training  Manual,  however, 
showed  that  while  the  manual  describes  overall  train¬ 
ing  needs,  as  well  as  tasks  which  could  be  taught  in  a 
simulator,  it  does  not  require  specific  tasks  to  be 
performed  in  the  simulator  during  the  required  20 
hours.  Furthermore,  Army  officials  at  the  locations 
visited  told  us  that  operational  commands  hav-t  no 
structured  syllabi. 
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Regarding  Defense’s  comments  about  the  Navy,  we 
recognize  on  page  5  that  operational  flight  trainers 
are  used  for  periodic  proficiency  training  and  quali¬ 
fication.  However,  neither  that  fact  nor  the  regula¬ 
tions  cited  by  defense  constitute  incorporation  of 
simulators  into  specific  training  syllabi  for  experi¬ 
enced  pilots.  In  fact,  only  one  cited  regulation  was 
relevant?  and  while  it  described  ways  simulators  could 
be  used,  it  left  the  use  up  to  the  squadron 
commander's  discretion. 

We  disagree  ^ith  Defense's  comment  that  formal 
syllabi  for  fleet  pilots  are  in  use  for  the  2E6  at 
Oceana,  Virginia.  The  only  2E6  syllabus  for  fleet 
pilots  at  Oceana  consists  of  three  lesson  plans  for 
exercises,  which  pilots  must  do  on  the  2E6  before  par¬ 
ticipating  in  a  specific  air  combat  maneuvering  exer¬ 
cise  with  actual  aircraft.  This  limited  application 
is  a  step  in  the  right  direction  but,  in  our  opinion, 
certainly  does  not  constitute  the  guidance  necessary 
for  a  device  as  costly  and  as  complicated  as  the  2E6. 

The  Air  Force  regulation  cited  requires  pilots  to 
train  in  simulators  12  hours  a  year,  not  the  18  hours 
Defense  mentioned  and  not  the  4  hours  referred  to  in 
our  draft.*  (We  have  changed  our  report  accordingly 
on  p.  5.)  Nevertheless,  our  point  was  that  while  mini¬ 
mum  hours  are  required,  specific  training  to  be  done 
during  those  hours  is  not  specified.  Although  Air 
Force  regulations  require  preparation  of  lesson 
plans,  operational  units  we  visited  said  that  use  of 
the  devices  was  left  up  to  the  pilot  and  no  lesson 
plans  were  provided. 

4.  The  Navy's  F-14  and  the  Air  Force's  F~15  operational 
flight  trainers  had  been  incorporated  into  their  training  course 
for  new  pilots  but  not  for  operating  units  where  only  minimum 
hour  requirements  for  simulator  use  was  established  such  as 
periodic  proficiency  qualifications  preparation  and  testing. 

(P9  3) 

DoD  Position:  Nonconcur.  F-14  and  F-15  operational  squadrons 
are  using  the  operational  flight  trainers  (OFT)  for  exactly  their 
designed  function.  The  OFT's  are  designed  to  provide  instruction 
and  refresher  in  navigation,  instrument,  take  off/landing  and 
normal/emergency  operating  procedures.  The  use  of  the  OFT  for 
proficiency  qualification  preparation  and  testing  for  experienced 
pilots  is  a  valid  use  of  the  training  medium. 
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GAO  Comment 

Although  Defense  disagrees  with  our  statement 
that  pilots  in  operating  units  were  required  to  use 
the  operational  flight  trainers  for  only  a  minimum 
number  of  hours,  the  information  it  presents  does  not 
disprove  our  data.  Defense  does  not  address  the  prob¬ 
lem  of  (1)  defining  which  tasks  can  best  be  taught  on 
the  devices  and  (2)  requiring  the  devices  to  be  used 
for  these  tasks.  Also,  Defense  does  not  mention  that 
the  devices  can  be  used  for  additional  tasks.  For 
example,  according  to  Navy  publications,  the  opera¬ 
tional  flight  trainers  for  the  F-14  can  be  linked  to  a 
radar  and  weapons  trainer  and  can  provide  team 
training  in  tactics. 

5.  The  Services  do  not  always  analyze  their  training  needs 
nor  incorporate  simulators  into  training  programs,  and  as  a 
result  they  are  left  with  systems  that  may  not  meet  their  needs 
and  that  may  not  be  used  as  effectively  as  they  could  be. 

(pgs.  5  and  6) 

DoD  Position;  Nonconcur.  As  discussed  above,  simulators  have 
been  incorporated  into  unit  training  programs  for  pilots  and  the 
Services  have  analyzed  their  training  needs.  As  a  result,  the 
Services  have  used  simulators  effectively  to  improve  pilot 
training  and  attain  higher  levels  of  readiness.  Limitations  in 
flight  simulator  capabilities  to  satisfy  some  pilot  training 
needs  has  been  constrained  by  technology,  not  a  failure  to 
analyze  training  needs. 

Navy  directives  addressed  in  response  to  Finding  .#2  apply.  -  These 
three  directives  provide  overall  guidance  for  the  development  of 
aviation  trainers  as  well  as  specific  responsibility  and 
authority  for  phases  of  the  planning,  programming,  budgeting  and 
acquisition  process.  The  development  of  every  major  aviation 
trainer  must  be  conducted  within  the  framework  of  these 
directives.  The  more  recent  addition  of  the  ISD  MIL-T-29053B 
(TD)  provides  a  more  specific  discipline  to  the  training 
requirements  and  has  introduced  the  concept  of  an  instructional 
system.  This  concept,  incorporated  in  the  training  device 
acquisition  process  in  1979,  has  provided  a  new  dimension  to  the 
training  analysis  conducted  by  the  Navy.  The  ISD  process  allows 
a  systems  approach  to  training.  This  allows  the  total  training 
(training  media,  hardware,  courseware,  etc.)  to  be  integrated 
into  a  training  syllabus.  The  integration  permits  the  proper 
phasing  or  placement  of  the  trainer  in  the  training  process.  To 
impose  current  acquisition  milestones  on  1972-1974  acquisition 
processes  is  inappropriate. 

GAO  Comment 

Defense  is  restating  our  conclusions  from  items 
#2,  #3,  ar.d  #4.  (Refer  to  item  #2  for  a  discussion  of 
service  directives  and  the  services'  use  of  the  ISD 
process. ) 
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6.  The  Navy's  2E6  air  combat  maneuvering  simulator  ($18 
million)  is  only  being  used  primarily  to  provide  new  pilots  basic 
air-to-air  tactics  training  while  it  was  designed  for  additional 
capabilities.  (pg.  5) 

DoD  Position:  Nonconcur.  In  the  particular  case  of  the  2E6,  the 
trainer  is  supposed  to  prepare  a  pilot  for  air  combat  maneuvering 
(ACM) ,  rather  than  act  as  a  substitute  for  airborne  ACM.  Pre¬ 
paration  requires  task-oriented  training  for  air  crews  on  the 
initial  phases  of  in-flight  air  combat  maneuvering  training.  The 
trainer  is  generally  used  up  to  the  limits  of  its  capability  to 
learn  valuable  procedures  which  make  airborne  time  much  more 
meaningful.  In  addition  to  new  pilot,  training,  fleet  pilots  use 
the  trainer  to  enhance  their  air-to-air  combat  maneuver  skills  in 
missile  envelope  recognition,  (1-v-l,  l-v-2,  and  2-v-l) ,  weapon 
selection  and  firing,  air-to-air  guns  against  both  maneuvering 
and  nonmaneuvering  targets,  offensive/defensive  maneuvering 
against  other  aircraft,  inter  and  intra  cockpit  communications, 
threat  familiarization,  safe  disengagements,  section  coordination 
in  multi  bogey  environments,  for  which  formal  syllabi  are  in  use 
at  COMFITWINGONE,  Oceana,  Va.  Copies  of  these  syllabi  were 
provided  to  the  GAO  team  during  their  visit.  Utilization  reports 
for  this  device  (3M,  COMFITWINGONE  and  FASODET,  Oceana  data)  for 
the  past  24  months  show  that  the  2E6  device  was  useu  48%  by  fleet 
squadron  pilots,  47%  by  Fleet  Readiness  Squadron  (FRS)  and  4%  by 
others. 

GAO  Comment 


Defense's  response  covers  three  areas:  substitu- 
tion,  tasks  being  performed  on  the  simulator  as  speci¬ 
fied  in  syllabi,  and  utilization  reports. 

First,  we  make  no  mention  in  our  report  of  sub- 
stit  ng  2E6  simulation  flights  for  flying  the  actual 

airci  t.  We  agree  that  time  in  the  2E6  should  make 
airborne  time  much  more  meaningful. 

We  do  not  agree  with  Defense's  statement  that  the 
Navy  is  using  the  2E6  to  the  limits  of  its  capability 
because  the  Navy  recently  contracted  to  determine  the 
2E6's  capability.  (See  p.  5.) 

Numerous  Navy  officials  at  headquarters  and  in 
the  field  told  us  that  it  was  unreasonable  to  expect 
the  fleet  squadrons  to  be  able  to  effectively  use  a 
device  as  complicated  as  the  2E6  with  no  structured 
lesson  plans  or  even  descriptions  of  what  the  device 
is  capable  of  doing.  The  only  syllabus  available  for 
fleet  pilots  is  the  one  described  in  item  #3  (which 
was  developed  2  years  after  the  2E6  was  fielded.) 

Even  doing  the  tasks  described  in  Defense's  response 
does  not  mean  the  simulator  is  being  used  to  the 
limits  of  its  capability,  since  the  tasks  listed  for 
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air  combat  maneuvering  could  represent  anything  from  a 
basic  introduction  to  very  difficult  situations. 

Although  Defense's  response  showed  48  percent 
usage  for  fleet  squadron  pilots  and  47  percent  for  new 
pilots,  our  data  shows,  and  Navy  officials  at  Oceana 
and  the  Atlantic  Air  Command  agreed,  that  at  any 
particular  time  only  about  10  new  pilots  are  eligible, 
while  46  fleet  pilots  are  available,  for  training  on 
the  2E6.  Because  total  utilization  of  the  device  for 
both  groups  is  about  equal,  evidently  new  pilots,  who 
are  required  to  learn  only  minimum  tactics  skills,  are 
receiving  far  more  training  on  the  2E6  than  fleet 
pilots. 

7.  The  2E6  was  also  scheduled  for  use  in  the  Navy's  Pacific 
air  command  but  the  Command  has  expressed  doubt  as  to  the  2E6 
effectiveness  and  believed  that  equivalent  training  could  be 
obtained  from  other  simulators.  As  a  result  the  Command  did  not 
receive  nor  request  the  2E6.  (pg.  5) 

DoD  Position:  Nonconcur.  The  Navy's  Pacific  Air  Command 
requested  that  modifications  be  incorporated  into  the  procurement 
of  the  second  2E6.  The  modifications  included  radar  and  full 
weapons  systems  capabilities  with  back  seat  crew  stations.  Since 
funding  was  not  available  to  effect  these  modifications,  AIRPAC 
redirected  its  requirements  to  modification  of  the  2F-112  with  a 
single  dome,  now  in  place.  This  trainer  does  not  provide  l-v-2 
or  2-v-l  tactical  air  crew  training  capabilities.  Since  funding 
to  incorporate  the  modifications  for  the  2E6  was  not  available, 
the  Navy  decided  to  rebudget  for  the  second  trainer  in  another 
/ear. 


GAO  Comment 


Defense  does  not  refute  our  statement,  but  merely 
provides  additional  information  needing  explanation. 
According  to  documents  at  the  Pacific  Air  Command,  the 
modifications  to  the  2E6,  which  the  command  requested, 
were  necessary  to  make  the  device  adequate  for  train¬ 
ing  air  combat  maneuvering  beyond  basic  levels.  The 
command  questioned  in  numerous  messages  to  Navy  Head¬ 
quarters  the  wisdom  of  funding  such  an  expensive 
device  which  would  not  meet  the  command's  require¬ 
ments. 

Also,  we  question  whether  adding  the  capability 
to  maneuver  against  or  with  a  third  aircraft  (the  dif¬ 
ference  cited  by  Defense  between  the  2F112  and  the 
2E6)  justifies  the  millions  the  2E6  will  cost.  Even 
if  funding  had  been  available,  we  believe  the  Navy 
would  have  been  ill-advised  to  install  a  2E6  for  the 
Pacific  Air  Command,  given  the  overwhelming  objection 
to  it,  and  especially  in  view  of  the  dearth  of  train¬ 
ing  materials  to  ensure  proper  use.  No  evidence  was 
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presented  to  show  that  the  Pacific  Air  Command  has 
reversed  its  opposition  to  the  2E6, 

8.  The  2P95  for  fleet  squadron  training  has  been  used  less 
than  50  percent  of  the  available  training  time  because  it  is  used 
mostly  for  pilots'  practice  before  flight  qualifications  tests  in 
actual  aircraft  and  has  not  been  made  part  of  the  squadron's 
training  program  because  it  did  not  specifically  meet  fleet 
squadron  training  needs.  (pg.  6) 

DoD  Position;  Nonconcur.  The  ,2F95  trainer  is  being  used  for  the 
purposes  for  which  it  was  intended.  It  does  not  meet  all 
squadron  training  needs  but  the  reason  is  limitations  of  1960's 
technology,  not  a  failure  to  properly  analyze  training  needs. 

It  has  limited  capabilities  by  design  but  is  an  excellent  trainer 
for  instruments,  navigation,  normal  and  emergency  system 
operating  procedures,  and  night  carrier  landing  training.  This 
device  is  incorporated  into  the  AIRLANT  F-14  Fleet  Squadron 
Training  programs  to  the  extent  of  its  capabilities  by  direction 
of  OPNAVINST  3710. 7K,  COMNAVAIRLANT  letter  Serial  0365  of 
9  January  1980,  and  FITWINGONEINSTR  3500.6. 

GAO  Comment 

Although  Defense  disagrees  with  our  statement 
about  use  of  the  2F95,  it  takes  no  exception  to  the 
amount  of  time  we  reported  the  device  was  used  or  how 
it  was  used.  Defense  points  out  the  limitations  of 
the  device  but  does  not  explain  why  all  the  features 
were  not  used,  in  addition,  Defense  states  that  the 
device  was  included  in  fleet  pilot  training  programs 
and  cited  Atlantic  Air  Command  documents;  however,  as 
stated  on  page  6,  the  2F95  we  examined  was  at  the 
Pacific  Air  Command,  not  the  Atlantic  Air  Command. 

9.  Service  regulations  require  that  training  needs  be 
analyzed  to  justify  buying  simulators  but  GAC  found  that  training 
needs  are  not  always  adequately  analyzed  to  form  the  criteria  for 
developing  new  simulators.  (pg.  6) 

DoD  Position :  Nonconcur .  As  stated  above  in  the  response  to 
Findings  #2  and  #5,  the  Services  do  analyze  their  training  needs 
to  form  the  basis  for  the  development  of  new  simulators.  The 
training  analysis  and  documentation  of  training  needs  for  all  new 
simulators  is  done  under  the  Instructional  Systems  Development 
process.  For  example,  new  simulator  programs  for  the  Army's 
AH-64,  Navy's  F/A-18,  and  Air  Force's  F-16  and  B-1B  have 
extensive  documentation. 

GAO  Comment 


This  statement  generally  repeats  item  2.  The 
only  new  information  in  Defense's  response  concerns 
several  new  simulator  programs,  none  of  which  we 
studied  and,  consequently,  cannot  comment. 
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10.  In  the  Navy,  two  simulators,  the  TH-57  (helicopters) 
and  the  EA-6B  cockpit  procedures  trainer  have  had  hardware 
decisions  made  without  training  needs  analyzed.  (pg.  6) 

DoD  Position:  Nonconcur.  Cockpit  procedures  trainers  such  as 
the  TH-$">  and  EA-6B  have  fairly  standardized  training  require¬ 
ments  and  the  Navy  has  much  experience  and  knowledge  regarding 
the  training  needs  to  be  satisfied.  The  Navy's  experience  and 
knowledge  on  training  needs  was  considered  as  a  part  of  the 
decision  regarding  these  trainers.  In  all  cases,  hardware 
decisions  concerning  a  trainer  are  reviewed  by  a  Fleet  Project 
Team  to  ensure  training  needs  ara  properly  evaluated.  Reviews 
are  conducted  by  the  Navy  in  accordance  with  the  documents  cited 
in  the  response  to  Finding  #2. 

GAO  Comment 


As  pointed  out  on  page  6,  our  source  for  the 
statement  that  hardware  decisions  on  the  TH-57  and 
EA-6B  were  made  without  training  needs  assessments  was 
a  Navy  official  from  the  organization  responsible  for 
training  equipment.  Nevertheless,  we  believe 
Defense's  reasons  for  not  assessing  training  needs, 
such  as  Navy  knowledge  and  review  by  a  Fleet  Project 
Team,  are  inadequate.  The  Fleet  Project  Team's 
review,  for  example,  is  more  a  roundtable  discussion 
than  a  rigorous  analysis  of  training  needs  and  the 
best  way  to  meet  them. 

11.  The  Navy  plans  to  buy  another  2E6  ($23.3  million)  which 
is  of  concern  because  training  needs  were  not  analyzed  for  the 
initial  trainer  and  incorporated  into  operating  units'  training 
programs,  also,  a  user  for  the  planned  2E6  has  not  been 
identified.  (pg.  6) 

DoD  Position:  Nonconcur.  Training  needs  were  well  identified  by 
the  Navy  for  the  2E6  Air  Combat  Maneuvering  (ACM)  simulator. 

(See  response  to  Findings  2,  5,  6,  7,  and  9.)  The  2E6  was 
conceived,  designed,  and  is  used  as  a  part-task  trainer 
specifically  to  prepare  aircrews  for  in-flight  Air  Combat 
Maneuvering  training.  The  first  2E6  trainer  is  incorporated  into 
fleet  squadron  training.  A  second  de'vice  is  programmed  for 
AIRPAC  to  fulfill  the  l-v-2  and  2-v-l  ACM  requirements  which  are 
not  present  in  the  2F112. 

GAO  Comment 

Although  the  need  for  an  air  combat  maneuvering 
simulator  was  espoused  repeatedly  arid  numerous  meet¬ 
ings  were  held  to  discuss  equipment  characteristics, 
we  saw  no  evidence  during  our  review  that  an  analysis 
of  tasks  which  needed  to  be  taught  was  developed  to 
support  the  equipment  capabilities,  and  Defense 
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provided  no  additional  evidence  that  any  such  analysis 
was  performed. 

Defense  also  states  that  the  2E6  is  incorporated 
into  fleet  squadron  training  and  that  a  user  exists 
for  an  additional  device.  Items  #2  and  #7  provide  our 
comments  on  these  issues.  The  Pacific  Air  Command's 
strong  objections  further  support  our  position  that  a 
user  for  an  additional  2E6  has  not  been  identified. 

CONCLUSIONS: 

1.  Simulators  are  becoming  an  increasingly  important 
element  of  military  training  but  they  are  becoming  more  costly, 
(pg.  7) 

DoD  Position:  Concur. 

2.  Added  benefits  from  their  large  investment  can  be 
achieved  if  the  Services  developed  simulators  to  meet  proven 
training  needs  and  incorporated  them  into  specific  training 
programs.  (pg.  9) 

DoD  Position :  Concur .  The  Services  conduct  extensive  reviews  ot 
their  simulator  programs  to  ensure  training  needs  are  justified. 
Through  the  Instructional  Systems  Development  (ISD)  process, 
simulators  are  incorporated  into  specific  training  programs. 

RECOMMENDATIONS :  That  the  Secretary  of  Defense . 

1.  Approve  budget  requests  for  flight  simulators  only  after 
the  Services  have  analyzed  their  training  needs  and  proven  that 
the  needs  cannot  be  met  with  existing  simulators.  (Specific 
review  should  be  made  of  the  pending  purchase  of  an  additional 
2E6.)  (pg.  7) 

DoD  Position:  Concur .  The  Services  analyze  and  review  their 
training  needs  as  a  part  of  the  preparation  for  the  program  and 
budget  review  process.  The  simulator  procurement  program  is 
reviewed  and  updated  each  year  during  the  program  cycle.  Budget 
decisions  for  all  new  simulators  are  made  subsequent  to  the 
completion  of  several  program  reviews. 

-2.  Require  the  Services  to  incorporate  simulatbrs  into 
their  training  programs,  (pg.  7) 

DoD  Position:  Concur.  The  Department  agrees  with  the 
recommendation  in  principle;  however,  the  incorporation  of 
simulators  into  the  training  plan  for  pilots  is  an  integral  part 
of  the  DoD  Instructional  Systems  Development  Process.  Further, 
each  of  the  Services  has  identified  specific  simulator  training 
that  is  required  for  pilots. 
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